System and method for providing complex haptic stimulation during input of control gestures, and relating to control of virtual equipment

ABSTRACT

A system is configured to provide haptic stimulation to a user. In one embodiment, the haptic stimulation is provided to the user in conjunction with the performance of one or more control gestures through which the user controls, for example, a game, a real world component or piece of equipment, and/or other entity. In one embodiment, the haptic stimulation is provided to the user in conjunction with control of virtual equipment by the user.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 12/840,797, filed Jul. 21, 2010, issued as U.S.Pat. No. 8,469,806 on Jun. 25, 2013, and entitled “System And Method ForProviding Complex Haptic Stimulation During Input Of Control Gestures,And Relating To Control Of Virtual Equipment,” which claims priorityfrom U.S. Provisional Patent Application No. 61/227,645, filed Jul. 22,2009, and entitled “Interactive Touch Screen Gaming Metaphors WithHaptic Feedback.” Both U.S. patent application Ser. No. 12/840,797 andU.S. Provisional Patent Application No. 61/227,645 are herebyincorporated by reference in their entireties into the presentapplication.

FIELD OF THE INVENTION

The invention relates to a system and method of providing hapticstimulation to a user during performance of a complex control gesture,and/or during the control of virtual equipment.

BACKGROUND OF THE INVENTION

Provision of haptic stimulation to users is known. Haptic stimulationprovide a physical sensation to users. Haptic stimulation is used in thecontext of games, and virtual worlds, and in real world control systems.Such haptic stimulation may be generated to provide feedback to usersthat a control input has been received, that another user has input acommand, that virtual or real objects have collided, exploded, orimploded, that an ambient force is present (e.g., simulated or realwind, rain, magnetism, and/or other virtual forces), and/or that otherphenomena have occurred. In conventional systems, the parameters of suchstimulation is typically static and provides a simple mechanism forinstructing a user that a corresponding phenomena has occurred (or willoccur).

Conventional game and/or virtual world systems that enable a user tocontrol virtual equipment are known. The control and feedback schemesfor interacting with virtual equipment in these conventional systemstend to be limited, and to not correlate strongly with real worldcontrol and/or feedback of corresponding real world equipment.

Although basic haptic effects (e.g., vibrate) have been used in mobileand other computing devices, numerous challenges remain for developersto engage users and provide feedback to enhance the user experience

SUMMARY

One aspect of the invention relates to a system configured to providehaptic stimulation to a user of a game. In one embodiment, the systemcomprises a user interface, an actuator, and one or more processorsconfigured to execute computer program modules. The user interface isconfigured to generate output signals related to the gestures of a user.The actuator is configured to generate haptic stimulation to the user.The computer program modules comprise a gesture module, a stimulationmodule and an actuator control module. The gesture module is configuredto monitor performance of a control gesture by the user based on theoutput signals of the user interface. The control gesture is a gestureassociated with a command input to the game, and includes an initialportion, a first intermediate portion, and an ending portion. Thestimulation module is configured to receive information related toperformance of the control gesture from the gesture module, and todetermine haptic stimulation to be generated for the user associatedwith the control gesture. The haptic stimulation includes a firststimulation determined responsive to performance of the initial portionof the control gesture, and a second stimulation that is different fromthe first stimulation and is determined responsive to performance of thefirst intermediate portion of the control gesture. The actuator controlmodule is configured to control the actuator to generate the stimulationdetermined by the stimulation module.

Another aspect of the invention relates to a method of providing hapticstimulation to a user of a game. In one embodiment, the method comprisesmonitoring performance of a control gesture by a user, wherein thecontrol gesture is a gesture associated with a command input to thegame, and includes an initial portion, a first intermediate portion, andan ending portion; determining haptic stimulation associated withperformance of the control gesture to be generated for the user, whereinthe haptic stimulation includes a first stimulation determinedresponsive to performance of the initial portion of the control gesture,and a second stimulation that is different from the first stimulationand is determined responsive to performance of the first intermediateportion of the control gesture; and generating the determinedstimulation during performance of the control gesture.

Yet another aspect of the invention relates to a system configured toprovide stimulation to a user of a game. In one embodiment, the systemcomprises a touch sensitive electronic display, an actuator, and one ormore processors configured to execute computer program modules. Thetouch sensitive electronic display has an interface surface that isaccessible for engagement by the user, and the touch sensitive userinterface is configured to generate output signals related to theposition at which the interface surface is engaged, and to present viewsof the game to the user through the interface surface. The viewspresented through the interface surface include views of virtualequipment having user selectable sections that are selectable by theuser to interact with the virtual equipment by engaging the interfacesurface at the user selectable sections of the views of the virtualequipment. The actuator is configured to generate haptic stimulation tothe user. The computer program modules comprise an equipment module, astimulation module, and an actuator control module. The equipment moduleis configured to determine the operating parameters of the virtualequipment in the views, and to simulate operation of the virtualequipment. The equipment module determines the operating parameters ofthe virtual equipment and/or simulates operation of the virtualequipment based on selections by the user of the user selectablesections of the views of the virtual equipment. The stimulation moduleis configured to determine haptic stimulation to be generated for theuser associated with the operating parameters of the virtual equipmentand/or simulated operation of the virtual equipment. The actuatorcontrol module is configured to control the actuator to generate thestimulation determined by the stimulation module.

Still another aspect of the invention relates to a method of providingstimulation to a user of a game. In one embodiment, the method comprisespresenting views of a game through an interface surface of a touchsensitive electronic display that is accessible for engagement by auser, wherein the views presented through the interface surface includeviews of virtual equipment having user selectable sections that areselectable by the user to interact with the virtual equipment byengaging the interface surface at the user selectable sections of theviews of the virtual equipment; receiving selection of one of the userselectable sections via an by the user engagement of the selected userselectable section on the interface surface; determining the operatingparameters of the virtual equipment in the views and/or simulatingoperation of the virtual equipment based on the received selection;determining, responsive to the received selection, haptic stimulation tobe generated for the user associated with the operating parameters ofthe virtual equipment and/or simulated operation of the virtualequipment; and generating the determined haptic stimulation.

A still further aspect of the invention relates to a system and methodfor providing a game on one or more portable computing device in which avirtual object (e.g., a ball) travels through views of the gamedisplayed on the interfaces of the one or more portable computingdevices. Haptic effects corresponding to the travel of the virtualobject (or virtual objects) through the views are provided on theindividual portable computing devices. The haptic effects may bedetermined based on one or more parameters of the travel of the virtualobject (e.g., speed, direction, acceleration, etc.), one or moreparameters of objects and/or features with which the virtual objectinteracts (e.g., walls, flippers, blockers, bumpers, etc.), and/or otherparameters. The haptic effects may include haptic effects to be providedon portable computing devices that are not currently displaying thevirtual object corresponding to the haptic effects. This may enhance theinteractivity of the game for a group of users playing the game togetheron separate portable computing devices.

These and other objects, features, and characteristics of the presentinvention, as well as the methods of operation and functions of therelated elements of structure and the combination of parts and economiesof manufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the invention. As usedin the specification and in the claims, the singular form of “a”, “an”,and “the” include plural referents unless the context clearly dictatesotherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system configured to provide haptic stimulation toa user, in accordance with one or more embodiments of the invention.

FIG. 2 illustrates a view of a piece of virtual equipment, in accordancewith one or more embodiments of the invention.

FIG. 3 illustrates a view of a piece of virtual equipment, in accordancewith one or more embodiments of the invention.

FIG. 4 illustrates a view of a piece of virtual equipment, in accordancewith one or more embodiments of the invention.

FIG. 5 illustrates a method of providing haptic feedback to a user,according to one or more embodiments of the invention.

FIG. 6 illustrates a method of providing haptic feedback to a user,according to one or more embodiments of the invention.

FIG. 7 illustrates a portable computing device, in accordance with oneor more embodiments of the invention.

FIG. 8A illustrates an example of the use of a game to supportmulti-user play, according to one or more embodiments of the invention.

FIG. 8B illustrates an example of the use of a game to supportmulti-user play, according to one or more embodiments of the invention.

FIG. 9 illustrates an example of play areas for two respective usersover a plurality of time intervals, in accordance with one or moreembodiments of the invention.

FIG. 10 illustrates an example of play and depicts the virtual objectricocheting off the border of a play area, according to one or moreembodiments of the invention.

FIG. 11 illustrate a use of haptic effects to simulate a continuouseffect, in accordance with one or more embodiments of the invention.

FIG. 12 illustrates a method for providing a game, according to one ormore embodiments of the invention.

FIG. 13 illustrates an example of an interface for an instance of agame, in accordance with one or more embodiments of the invention.

DETAILED DESCRIPTION

FIG. 1 illustrates a system 10 configured to provide haptic stimulationto a user 12. In one embodiment, the haptic stimulation is provided touser 12 in conjunction with the performance of one or more controlgestures through which user 12 controls, for example, a game, a realworld component or piece of equipment, and/or other entity. The hapticstimulation is provided to user 12 such that as performance of a givencontrol gesture continues, the haptic stimulation associated with thecontrol gesture changes in accordance with the progression of thecontrol gesture. In one embodiment, the haptic stimulation is providedto user 12 in conjunction with control of virtual equipment by user 12.The haptic stimulation corresponds to control inputs provided to system10 in controlling the virtual equipment. In one embodiment, system 10includes one or more user interfaces 14, one or more actuators 16,electronic storage 18, one or more processors 20, and/or othercomponents.

Although this disclosure primarily describes the provision of hapticstimulation in conjunction with control gestures performed to control agame, this is not limiting. The provision of haptic stimulation inaccordance with the principles set forth herein may be extended to othercontexts in which a user provides input in the form of a control gesture(e.g., controlling a television, a computer, an automobile, a remotecontrol vehicle or vessel, etc.). In implementations involving thecontrol of real world components or systems, haptic stimulation may beused to enhance the performance of complex control gestures by users,enhance the user experience, teach users to perform control gestures,and/or provide other enhancement over conventional real world controlsystems. Implementations involving control of real world components orsystems may include instances in which a user is present at thecomponent being controlled (e.g., a control panel in a car, on amicrowave, etc.), and/or instances in which a user is located remotelyfrom the component being controlled. Control of a remote real worldcomponent or system may be accompanied by other sensory stimulationinforming the user of the state of the component being controlled. Suchother sensory stimulation may include, for example, real-time (or nearreal-time) video, audio, and/or still images provided to the user.

Several embodiments are described herein as though user 12 were the onlyuser for whom haptic stimulus is being provided. This is not limiting.Expansion of the principles and embodiments described herein is withinthe ability of one of ordinary skill in the art, and the description ofa single user embodiment would enable the person of ordinary skill inthe art to make and/or user a multiple user embodiment providing thesame features to a plurality of users. A multiple user embodiment mayinclude the provision of haptic stimulation to “passive users” (e.g.,users not performing a control gesture) related to the performance of acontrol gesture by an “active user” (e.g., the user performing thecontrol gesture). The haptic stimulation may further provide feedback tothe passive and/or active users of other phenomena present in themulti-user environment.

As used herein, a “control gesture” refers to a gesture made by a userthat is a single and discrete control input having separate portions.The separate portions must be performed in specific order and/or with aspecific timing to effectively achieve the control input associated withthe “control gesture.” Performance of the separate portions, on theirown, will not result in the control input associated with the “controlgesture” as a whole (e.g., a “control gesture” is not merely acombination of other gestures, each associated with its own controlinput). In some examples, a “control gesture” is an abstract gesturethat does not correlate with exactness to the control input to which itcorresponds. Some non-limiting examples of a “control gesture” aredescribed below.

The user interface 14 includes one or more input and/or output devicesconfigured to communicate information to and/or receive information fromuser 12. The user interface 14 may include, for example, one or morecontent delivery devices that convey content to user 12. This contentmay include audio content, video content, still images, and/or othercontent. The content delivery devices may include, for example,electronic displays (e.g., including touch-sensitive displays), audiospeakers, and/or other content delivery devices. The user interface 14may include one or more control input devices configured to generate anoutput signal indicating input from the user to system 10. For example,user interface 14 may include a game controller, a remote control, akeypad, a button, a switch, a keyboard, a knob, a lever, a microphone, aposition detecting device (e.g., an image sensor, a pressure sensor, anoptical position detector, an ultrasonic position detector, a touchsensitive surface, and/or other position detecting devices), anaccelerometer, a gyroscope, a digital compass, and/or other controlinput devices. The user interface 14 may be embodied in, or associatedwith, a computing device, and/or may be embodied in a controlperipheral. A computing device may include one or more of a desktopcomputer, a laptop computer, a handheld computer, a personal digitalassistant, a Smartphone, a personal music player, a portable gamingconsole, a gaming console, and/or other computing devices.

It will be appreciated that although user interface 14 is shown in FIG.1 as a single entity, this is for illustrative purposes only. In oneembodiment, user interface 14 includes a plurality of actuators. Theplurality of user interfaces 14 may be included in, carried by, and/orin contact with a single object or device. Or, the plurality of userinterfaces 14 may include user interfaces included in and/or carried bya plurality of separate objects or devices.

The actuators 16 are configured to generate haptic stimulus for user 12.As such, at least some of actuators 16 are in contact with the users, orin contact with objects that contact the users, during conveyance of thesensory content to the users by user interface 14. By way ofnon-limiting example, one or more of actuators 16 may be positioned inor on a floor surface supporting the users (e.g., installed in thefloor, carried by a mat lying on the floor, etc.), one or more ofactuators 16 may be carried by a brace or other wearable item worn bythe users, one or more of the actuators 16 may be carried by objectsthat are carried by the users (e.g., carried by a controller), one ormore of actuators 16 may be carried by furniture on which the users areseated or lying, one or more of actuators 16 may be carried by userinterface 14, and/or one or more of the actuators 16 may be carried byor disposed in or on other objects that contact the users.

As used herein, the term “haptic stimulus” refers to tactile feedbackthat is applied to the users. For example, such feedback may include oneor more of vibrations, forces, and/or motions that are appliedphysically to the user by the actuators 16 and/or the objects with whichboth actuators 16 and the user are in contact. The actuators 16 mayinclude any device configured to generate such feedback for applicationto the users. For example, actuators 16 may include one or more of apiezoelectric actuator, a pneumatic actuator, a central mass actuator,an electroactive polymer actuator, an electrostatic surface actuator,macro-fiber composite actuator, and/or other actuators. For example, atouch sensitive surface (and/or other surfaces) may be actuated to moverelative to a user by an electrostatic actuator. The surface actuated byactuators 16 may be rigid, semi-rigid, flexible, and/or deformable.

It will be appreciated that although actuators 16 are shown in FIG. 1 asa single entity, this is for illustrative purposes only. In oneembodiment, actuators 16 include a plurality of actuators. The pluralityof actuators may be included in, carried by, and/or in contact with asingle object or device. Or, the plurality of actuators may includeactuators included in, carried by, and/or in contact with a plurality ofseparate objects or devices.

In one embodiment, electronic storage 18 comprises electronic storagemedia that electronically stores information. The electronic storagemedia of electronic storage 18 may include one or both of system storagethat is provided integrally (i.e., substantially non-removable) withsystem 10 and/or removable storage that is removably connectable tosystem 10 via, for example, a port (e.g., a USB port, a firewire port,etc.) or a drive (e.g., a disk drive, etc.). Electronic storage 18 mayinclude one or more of optically readable storage media (e.g., opticaldisks, etc.), magnetically readable storage media (e.g., magnetic tape,magnetic hard drive, floppy drive, etc.), electrical charge-basedstorage media (e.g., EEPROM, RAM, etc.), solid-state storage media(e.g., flash drive, etc.), and/or other electronically readable storagemedia. Electronic storage 18 may store software algorithms, informationdetermined by processor 20, information received via user interface 14,and/or other information that enables system 10 to function properly.Electronic storage 18 may be a separate component within system 10, orelectronic storage 18 may be provided integrally with one or more othercomponents of system 10 (e.g., processor 20).

Processor 20 is configured to provide information processingcapabilities in system 10. As such, processor 20 may include one or moreof a digital processor, an analog processor, a digital circuit designedto process information, an analog circuit designed to processinformation, a state machine, and/or other mechanisms for electronicallyprocessing information. Although processor 20 is shown in FIG. 1 as asingle entity, this is for illustrative purposes only. In someimplementations, processor 20 may include a plurality of processingunits. These processing units may be physically located within the samedevice, or processor 20 may represent processing functionality of aplurality of devices operating in coordination.

As is shown in FIG. 1, processor 20 may be configured to execute one ormore computer program modules. The one or more computer program modulesmay include one or more of a content module 22, a gesture module 24, astimulation module 26, an equipment module 28, an actuator controlmodule 30, and/or other modules. Processor 20 may be configured toexecute modules 22, 24, 26, 28, and/or 30 by software; hardware;firmware; some combination of software, hardware, and/or firmware;and/or other mechanisms for configuring processing capabilities onprocessor 20.

It should be appreciated that although modules 22, 24, 26, 28, and 30are illustrated in FIG. 1 as being co-located within a single processingunit, in implementations in which processor 20 includes multipleprocessing units, one or more of modules 22, 24, 26, 28, and/or 30 maybe located remotely from the other modules. The description of thefunctionality provided by the different modules 22, 24, 26, 28, and/or30 described below is for illustrative purposes, and is not intended tobe limiting, as any of modules 22, 24, 26, 28, and/or 30 may providemore or less functionality than is described. For example, one or moreof modules 22, 24, 26, 28, and/or 30 may be eliminated, and some or allof its functionality may be provided by other ones of modules 22, 24,26, 28, and/or 30. As another example, processor 38 may be configured toexecute one or more additional modules that may perform some or all ofthe functionality attributed below to one of modules 22, 24, 26, 28,and/or 30.

The content module 22 is configured to control the provision of contentto user 12 via user interface 14. If the content includes computergenerated images (e.g., in a game, virtual world, simulation, etc.),content module 22 is configured to generate the images and/or views fordisplay to user 12 through user interface 14. If the content includesvideo and/or still images, content module 22 is configured to access thevideo and/or still images and to generate views of the video and/orstill images for display on user interface 14. If the content includesaudio content, content module 22 is configured to generate theelectronic signals that will drive user interface 14 to output theappropriate sounds. The content, or information from which the contentis derived, may be obtained by content module 22 from electronic storage18.

In one embodiment, the content provided by content module 22 is contentassociated with a game. In this embodiment content module 22 isconfigured to render views of the game for display to user 12 via userinterface 14. The content module 22 further provides audio associatedwith the views in accordance with the machine readable-program codeassociated with the game.

The gesture module 24 is configured to receive one or more outputsignals generated by user interface 14 indicating control inputsreceived from user 12. Based on the received one or more output signals,gesture module 24 monitors performance of one or more control gesturesby user 12. In one embodiment, a control gesture being monitored bygesture module 24 includes an initial portion, one or more intermediateportions, and an ending portion.

In an embodiment in which user interface 14 includes a touch-sensitivesurface through which input is received from user 12, an initial portionof a control gesture may include initiating contact with thetouch-sensitive surface at one or more locations. The control gesturemay dictate a location (or locations for a multi-touch control gesture)at which contact is initiated, a pressure with which contact isinitiated, and/or other parameters of the initial contact between user12 and the touch sensitive surface. If user 12 is already in contactwith the touch-sensitive surface, the initial portion of the controlgesture may include moving to one or more locations corresponding to thecontrol gesture, maintaining contact (at specific location(s), orgenerally) with the touch-sensitive surface, motioning in one or morespecific directions, making one or more specific shapes, ending contactat one or more locations on the touch-sensitive surface, and/or otheractions. The one or more intermediate portions of the control gesturemay include one or more of maintaining contact at one or more pointswithout moving, motioning in a specific direction, making a specificshape, halting motion, contacting the touch-sensitive surface at one ormore additional locations, ending contact at one or more locations onthe touch-sensitive surface, pressing harder or softer on thetouch-sensitive surface, and/or other actions. The ending portion mayinclude one or more of ending contact at one or more locations on thetouch-sensitive surface, halting motion at one or more locations,motioning in a specific direction, making a specific shape, contactingthe touch-sensitive surface at one or more additional locations,pressing harder or softer on the touch-sensitive surface, and/or otheractions.

One or more of the actions dictated by the initial portion, the one ormore intermediate portions, and/or the ending portion of the controlgesture that are location-based may be associated with static locations(e.g., at the same location every time), dynamic locations that do notchange during the corresponding portion (e.g., the location may movebetween performances of the control gesture, but remains fixed while theportion corresponding to the location is performed), dynamic locationsthat change during the corresponding portion, and/or other types oflocations.

In the embodiment in which user interface 14 includes a touch-sensitivesurface, one of the most simple examples of a control gesture wouldinclude an initial portion of in which user 12 contacts thetouch-sensitive surface. The intermediate portion may include holdingthe contact made during the initial portion of the control gesture. Theending portion may include removing the contact made during the initialportion of the control gesture, and maintained during the intermediateportion of the control gesture.

In an embodiment in which user interface 14 includes one or more sensorsconfigured to monitor motion of user 12 in space (e.g., an imagingsensor, a pressure sensor, an accelerometer, and/or other sensors), aninitial portion of a control gesture may include one or more of facing aspecific direction, motioning or moving in a specific direction,motioning with a specific one or more appendages, positioning body partswith respect to each other in a specific manner (e.g., holding hands ina predetermined configuration, and/or other configurations of bodyparts), motioning or moving in a specific shape, motioning for aspecific amount of time, motioning or moving at (or above, or below) aspecific rate and/or acceleration), changing a direction of motion ormovement in a specific manner, and/or other actions. The one or moreintermediate portions may include one or more of changing orientation ofthe head, and/or other body parts to a specific direction, motioning ormoving in a specific direction, motioning with a specific one or moreappendages, motioning with body parts to move in a specific relationshipto each other, motioning or moving in a specific shape, motioning for aspecific amount of time, motioning or moving at (or above, or below) aspecific rate and/or acceleration), changing a direction of motion ormovement in a specific manner, halting motion or movement by one or morebody parts, and/or other actions. The ending portion of the controlgesture may include may include one or more of changing orientation ofthe head, and/or other body parts to a specific direction, motioning ormoving in a specific direction, motioning with a specific one or moreappendages, motioning with body parts to move in a specific relationshipto each other, motioning or moving in a specific shape, motioning for aspecific amount of time, motioning or moving at (or above, or below) aspecific rate and/or acceleration), changing a direction of motion ormovement in a specific manner, halting motion or movement by one or morebody parts, and/or other actions.

The gesture module 24 is configured to monitor performance of thecontrol gestures by obtaining the output signal of user interface 14indicating movement and/or motion of user 12, and comparing the movementand/or motion of user 12 with the control gestures. One or more of thecontrol gestures may be a function of the content being conveyed to user12 via user interface 14 by content module 22. For example, one or moreof the control gestures may be a function of a game being conveyed touser 12 via user interface 14. One or more of the control gestures maybe independent from the content being provided to user 12 via userinterface 14. For example, one or more of the control gestures maycontrol function of user interface 14, processor 20, and/or othercomponents.

The stimulation module 26 is configured to receive information relatedto performance of control gestures from gesture module 24, and todetermine haptic stimulation to be generated for user 12 associated withthe control gestures. The haptic stimulation determined by stimulationmodule 26 includes haptic stimulation that is responsive to performanceof the control gestures separate from the context in which they areperformed. For example, in the context of a game haptic stimulationresponsive to performance of a control gesture includes hapticstimulation that is not dependent on other variables within the gameother than performance of the control gesture.

The haptic stimulation determined by stimulation module 26 is complexand rich beyond the haptic stimulation typically associated with controlgestures. For example, in conventional systems, haptic stimulationassociated with control gestures tends to include a single stimulationthat is provided during and/or after a control gesture. This hapticstimulation provides confirmation of the control gesture. The hapticstimulation determined by stimulation module 26, by contrast, tracksmore closely with performance of the control gesture to increase theimmersive experience provided by the haptic stimulation.

In one embodiment, the haptic stimulation determined by stimulationmodule 26 corresponding to a control gesture includes a firststimulation, a second stimulation, and/or other stimulations. The firststimulation is different from the second stimulation. This means thatone or more parameters of the first stimulation is different from thesecond stimulation. The one or more parameters may include one or moreof, for example, periodicity, force, directionality, location, and/orother parameters of haptic stimulation. Variation of the one or moreparameters between the first stimulation and the second stimulation(and/or other stimulations) may be smooth, and/or may be discrete so asto create a distinct step in the parameter(s) of haptic stimulationbetween the first stimulation and the second stimulation.

The stimulation module 26 is configured to correlate provision of thefirst stimulation and the second stimulation with performance of thecontrol gesture to which the first stimulation and the secondstimulation correspond. In one embodiment, the first stimulation isdetermined by stimulation module 26 responsive to performance of theinitial portion of the control gesture and the second stimulation isdetermined by stimulation module 26 responsive to performance of one ormore intermediate portions of the control gesture. Responsive to theending portion of the control gesture, stimulation module 26 maydetermined another stimulation (e.g., a third stimulation), or may ceasethe provision of haptic stimulation associated with the control gesture.

In one embodiment, stimulation module 26 determines haptic stimulationassociated with a control gesture such that the stimulation is differentbetween intermediate portions of the control gesture. This may result ina smooth changes in the haptic stimulation as performance of the controlgesture proceeds, and/or in discrete changes in the haptic stimulationas performance of the control gesture transitions between intermediateportions.

The haptic stimulation determined by stimulation module 26 may beprovided to give user 12 some feedback about performance of a controlgesture. This may include stimulation that informs user 12 that aportion of the control gesture has begun and/or is currently ongoing,stimulation that prompts user 12 to begin a next portion of the controlgesture, and/or stimulation that provides other information aboutperformance of the control gesture to user 12.

In one embodiment stimulation module 26 provides haptic stimulation touser 12 that indicates to user 12 that performance of a control gesturehas failed. In this embodiment, responsive to failure by user 12 toperform a portion of a control gesture, or failing to properlytransition from one portion of the control gesture to another portion ofthe control gesture, stimulation module 26 determines a stimulation forprovision to user 12 that is indicative of the failure. For example, thefailure stimulation may include a fizzle, and/or other stimulation.

Some illustrative examples of control gestures and corresponding hapticstimulation are presented hereafter. The examples given correspond tosome classes of game characters commonly found in games that includecombat between players and/or between players and non-player characters.Specifically, the examples given below correspond to “casters”, “closecombat warriors”, and “ranged combat warriors.” It will be appreciatedthat these classes of characters are not limiting. Other classes ofcharacters, and/or hybrids included characteristics of a plurality ofthe classes, may be implemented in accordance with the principles setforth herein.

A “caster” is a character that casts spells (or other similar attacksand/or defenses) during combat. Examples of caster characters includewizards, priests, warlocks, engineers, and/or other characters. Tocontrol a caster to cast a given spell, user 12 may be required toperform a control gesture that corresponds to the given spell. As user12 progresses through the portions of the control gesture, the hapticstimulation determined by stimulation module 26 corresponding to thecontrol gesture may inform user 12 as to the progress of the controlgesture. The haptic stimulation associated with the control gesture mayfurther provide information about the power of the spell being cast(e.g., greater the force and/or more rapid periodicity may indicate amore powerful spell).

In one embodiment, as the control gesture proceeds between the portionsof the control gesture, the force and/or rapidity of the hapticstimulation determined by stimulation module 26 may increase. Thisincrease may be gradual as the control gesture proceeds and/or indiscrete steps (e.g., at transitions between the portions of the controlgesture). This is not intended to be limiting, as other parameters ofthe haptic stimulation may be changed to indicate the progress of thecontrol gesture. The haptic stimulation may provide feedback about the(thus far) successful progress of the control gesture, and/or promptuser 12 with respect to future performance of the control gesture (e.g.,to proceed to the next portion, to maintain the current portion, etc.).Upon performing the final portion of the control gesture, stimulationmodule 26 may determine haptic stimulation that indicates the spellbeing successfully cast. In one embodiment, user 12 may perform portionsof the control gesture that effectively “store” the spell for discharge.This may store the spell for an indefinite period, or for somepredetermined maximum storage time. During storage, stimulation module26 may determine haptic stimulation that confirms the ongoing storage ofthe spell. If the storage period has a maximum storage time, the hapticstimulation may indicate the status of the storage period with respectto the maximum storage time.

A “close combat warrior” is a character configured to fight enemies atclose quarters (e.g., hand-to-hand combat). Such a character may bearmed with swinging weapons (e.g., clubs, maces, etc.), stabbing and/orslashing weapons (e.g., knife, axe, sword, etc.), and/or other closequarter weapons. As user 12 controls the character to, for example,wield a blow, stimulation module 26 may determine haptic stimulationthat mimics some real-world properties of such attacks. For example, asuser 12 strikes into an opponent or object, a first feedback may mimicthe striking and/or cutting sensation. As user 12 withdraws a weaponafter such a blow, a second feedback may mimic the withdrawal of theweapon from the object or opponent. As another example, as user 12swings a weapon in a ready position (e.g., swinging a ball on a chain) afirst feedback determined by stimulation module 26 may mimic the realworld sensation that would be the result of such an activity in reallife. Releasing the weapon from the ready position (e.g., striking anopponent or object with the swinging ball) may result in thedetermination by stimulation module 26 of a second stimulation thatmimics the feel of such an attack. The intensity of an attack may bedetermined based on an amount of time a control gesture (or some portionthereof) is performed, the range of motion of a control gesture, a speedof motion during a control gesture, pressure of the contact between theuser and a touch screen included in user interface 14, and/or otherparameters of a control gesture (or some portion thereof).

In some implementations, the user interface 14 includes a touch screenin which displays of the content associated with a game is presented bycontent module 22. In such implementations, the location at which thecontrol gesture is made impacts the manner in which the game iscontrolled and/or the haptic stimulation determined by stimulationmodule 26 (for the user inputting the gesture and/or other users). Forexample, in controlling a close combat warrior, a user may input anattacking control gesture in a manner that indicates a specific portionof an enemy at which the attack should be directed (e.g., a leg, an arm,the head, the torso, etc.). Similarly, in controlling a close combatwarrior, a user may input a defense or blocking control gesture in amanner that indicates a specific portion of a character being controlledthat should be defended or blocked (e.g., a leg, an arm, the head, thetorso, etc.). This location-based aspect of the control gesture may betaken into account in determining the success and/or impact of thecorresponding control, and/or in determining the haptic stimulation thatcorresponds to the control gesture. Multiple attacks may be triggered bytouching multiple locations on the touch screen of user interface 14.

A “ranged combat warrior” is a character that is configured and armedfor attacking enemies from a range. This may include characters armed,for example, to release projectiles such as arrows, stones, bullets,rockets, and/or other projectiles. By way of non-limiting example, user12 may control a ranged combat warrior to fire a bow and arrow. In thisexample, different haptic stimulation may be determined by stimulationmodule 26 for notching the arrow, drawing the bow (and varying thehaptic stimulation to indicate increasing string tension), releasing thestring, the bowstring striking the forearm of the character uponrelease, and/or other portions of the control gesture associated withfiring an arrow. The intensity and/or speed of an attack or projectilemay be determined based on an amount of time a control gesture (or someportion thereof) is performed, the range of motion of a control gesture,a speed of motion during a control gesture, pressure of the contactbetween the user and a touch screen included in user interface 14,and/or other parameters of a control gesture (or some portion thereof).

The target at which a ranged attack is directed and/or the path of aprojectile emitted as part of a ranged attack may be determined based ona control gesture. For example, if user interface includes a touchscreen, the user performing a control gesture may contact a portion ofthe touch screen on which the target of the ranged attack is displayed(e.g., as part of the control gesture). Similarly, the user may trace onthe touch screen a path of a projectile emitted during a ranged attack.The tracing of the path of the projectile may form at least part of thecontrol gesture initiating the attack. The target and/or the pathindicated by the control gesture may impact the determination of thecorresponding haptic stimulation determined by stimulation module 26.

Game content may include boxes, rooms, gates, vehicles, and/or otherobjects or items that are “locked”, and must be opened throughperformance of some control gesture. In some implementations, inresponse to a user selecting such an object or item, stimulation module26 may determine haptic stimulation indicating to the user that theselected object or item is locked. Performance by the user of a controlgesture to unlock the object or item may result in the determination ofhaptic stimulation by stimulation module 26 corresponding to the controlgesture.

In one embodiment, the haptic stimulation determined stimulation module26 for a given control gesture may vary based on characteristics of acharacter in a game being controlled by user 12. For example, the hapticstimulation may vary as a function of character skill, characterfatigue, character injury, weapon type, weapon disrepair, and/or othercharacteristics.

Changing the haptic stimulation based on character fatigue and/or injurymay be a way of handicapping user 12 as a character that was fatigued orinjured would be. This may include altering the force, periodicity,and/or other parameters of the haptic stimulation to inhibit precisecontrol by user 12 (e.g., by increasing the force), to inhibitperception of cues and prompts in the haptic stimulation (e.g., byreducing force, increasing the force of portions of the hapticstimulation that do not provide cues and prompts, reducing response timeto cues and/or prompts, etc.), and/or otherwise handicapping user 12.

Varying the haptic stimulation based on character skill may provide agradual and/or graduated “unlocking” of skills and abilities. For acharacter with more training and/or skill in a given ability,stimulation module 26 may determine haptic stimulation that providesmore clear and/or more easily followed guide to performing a controlgesture corresponding to the given ability. For example, as the skill ofthe character increases, the haptic stimulation determined bystimulation module 26 for the character may provide more definite cuesand/or prompts (e.g., see the examples above with respect to characterfatigue) that guide user 12 through the control gesture. This enablesuser 12 to utilize the control gesture without being fully trained inthe corresponding ability, but may impact the reproducibility of thecontrol gesture for the user unless further training and/or otherskill-building is sought.

Altering the haptic stimulation determined for a control gesture basedon the equipment of a character may incentivize maintaining repair ofequipment and/or obtaining upgraded equipment without making suchactivities mandatory. Instead of a digital determination of whether acontrol gesture is available to user 12, the stimulation module 26provides haptic stimulation that makes the control gesture easier and/ormore enjoyable with “better” virtual equipment.

The equipment module 28 is configured to provide control over virtualequipment for user 12. Within the context of a game being provided touser 12 by system 10, user 12 may control virtual equipment (e.g.,directly or through a character that is using the virtual equipment. Insuch cases, content module 22 provides to user 12, via user interface 14views of the virtual equipment. The equipment module 28 is configured todetermine the operating parameters of the virtual equipment, and tosimulate its operation.

In one embodiment in which user interface 14 includes a touch sensitiveelectronic display, equipment module 28 determines the operatingparameters of the virtual equipment and/or simulates operation of thevirtual equipment based on selections by user 12 of user selectablesections of the views of the virtual equipment. The user selectablesections of the virtual equipment may be located in the views of thevirtual equipment that correspond to sections of the virtual equipmentanalogous to sections of equipment that would be engaged in real life toconfigure operating parameters of equipment and/or to operate equipment.The operating parameters of the virtual equipment that are configurableby engaging user selectable sections of the views of the virtualequipment include one or more of an amount of loaded rounds ofammunition, a level of disrepair, a stopping power, a projectilevelocity, an ammunition type, a noise level, and/or other operatingparameters.

By way of illustration, FIG. 2 shows a view of a piece of virtualequipment 32 (e.g., a gun). In the view shown in FIG. 2, piece ofvirtual equipment 32 includes a plurality of selectable sections 34(illustrated in FIG. 2 as section 34 a, section 34 b, section 34 c, andsection 34 d). By selecting section 34 a, user 12 may configure thehandle/stock of piece of virtual equipment 32. By selecting section 34b, user 12 may configure the ammunition of piece of virtual equipment32. By selecting section 34 c, user 12 may configure the sights of pieceof virtual equipment 32. By selecting section 34 d, user 12 mayconfigure the barrel/muzzle of piece of virtual equipment 32.

Selection of one of selectable sections 34 to configure piece of virtualequipment 32 may include simply selecting the desired selectable section34, and then selecting a new configuration for the selected section 34.In one embodiment, reconfiguration is made through a touch and drag typeof interaction. For example, user 12 may engage an area on userinterface 14 that corresponds to ammunition and “drag” the ammunition tosection 34 b to reload piece of virtual equipment 32. Similarly, user 12may select and drag a silencer (and/or other muzzle or barrel feature)to section 34 d of piece of virtual equipment 32 to change the soundcharacteristics of piece of virtual equipment 32 (and/or othercharacteristics).

As another illustrative example, FIG. 3 shows a view of a virtualfirearm 36 having a plurality of selectable sections 38 (shown in FIG. 3as section 38 a, section 38 b, section 38 c, and section 38 d). Theselectable sections 38 of stimulation module 26 may be selected in afashion similar to the one discussed above with respect to FIG. 2.

As yet another illustrative example, FIG. 4 shows a view of a piece ofvirtual equipment 40 configured to show the proximity of various itemsto a user (or to a character being controlled by a user) in a game. Thepiece of virtual equipment 40 includes a plurality of selectablesections 42 (shown in FIG. 4 as section 42 a, section 42 b, section 42c, section 42 d, and section 42 e). Individual ones of selectablesections 42 correspond to individual modes of piece of virtual equipment40 such that selection of one of selectable sections 42 results in pieceof virtual equipment 40 operating in accordance with the selected mode.For example, in a first mode piece of virtual equipment 40 may providean indication of the proximity of other characters, in a second modepiece of virtual equipment 40 may provide an indication of the proximityof enemies, in a third mode piece of virtual equipment 40 may provide anindication of the proximity of one or more resources (e.g., gold, food,ammunition, power-ups, etc.).

Returning now to FIG. 1, in one embodiment, stimulation module 26 isconfigured to determine haptic stimulation to be generated for the userthat is associated with the operating parameters of the virtualequipment and/or the simulated operation of the virtual equipment. Thismay include, for example, varying the haptic stimulation based on thecurrent operating parameters of a piece of virtual equipment,determining haptic stimulation during the reconfiguration of a piece ofvirtual equipment to reflect changes being made in the piece of virtualequipment, and/or other aspects of configuration and/or operation ofvirtual equipment.

By way of non-limiting example, if a virtual firearm is reloaded by user12, stimulation module 26 determines haptic feedback indicating thereload. The new ammunition is different from previously used ammunition,stimulation module 26 determines haptic feedback upon firing of thefirearm that is different from the haptic stimulation determined duringfiring of the previously used ammunition.

In one embodiment, actuators 16 are configured to actuate the touchsensitive electronic display that presents the views of the virtualequipment to user 12. In this embodiment, delivery of the hapticstimulation determined by stimulation module 26 to user 12 as user 12engages the touch sensitive electronic display results in a moreimmersive virtual equipment control experience. For example, as user 12engages the touch sensitive display to reload a gun, or to switch out apiece of equipment, haptic stimulation is provided to user 12 throughthe touch sensitive surface that corresponds to the control beingexerted over the gun.

The actuator control module 30 is configured to control actuators 16 togenerate the haptic stimulus determined by stimulation module 26. Thisincludes communicating the haptic stimulus to be generated fromprocessor 22 to actuators 16. The haptic stimulus to be generated may becommunicated over wired communication links, wireless communicationlinks, and/or other communication links between processor 22 andactuators 16.

FIG. 5 illustrates a method 44 of determining and/or providing hapticstimulation to a user. The haptic stimulation may be determined and/orprovided to the user in conjunction with a game, and/or in othercontexts. The operations of method 44 presented below are intended to beillustrative. In some embodiments, method 44 may be accomplished withone or more additional operations not described, and/or without one ormore of the operations discussed. Additionally, the order in which theoperations of method 44 are illustrated in FIG. 5 and described below isnot intended to be limiting.

In some embodiments, method 44 may be implemented in one or moreprocessing devices (e.g., a digital processor, an analog processor, adigital circuit designed to process information, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information). The one or moreprocessing devices may include one or more devices executing some or allof the operations of method 44 in response to instructions storedelectronically on an electronic storage medium. The one or moreprocessing devices may include one or more devices configured throughhardware, firmware, and/or software to be specifically designed forexecution of one or more of the operations of method 44.

At an operation 46, performance of an initial portion of a controlgesture by a user is monitored. Performance of the initial portion ofthe control gesture may be monitored based on the output signal of auser interface similar to or the same as user interface 14 (shown inFIG. 1 and described above). In one embodiment, operation 46 isperformed by a gesture module similar to or the same as gesture module24 (shown in FIG. 1 and described above).

At an operation 48, responsive to performance of the initial portion ofthe control gesture, a first haptic stimulation is determined. The firsthaptic stimulation corresponds to the initial portion of the controlgesture. In one embodiment, operation 48 is performed by a stimulationmodule 26 similar to or the same as stimulation module 26 (shown in FIG.1 and described above).

At an operation 50, the first haptic stimulation is generated for theuser. In one embodiment, operation 50 is performed by one or moreactuators similar to or the same as actuators 16 (shown in FIG. 1 anddescribed above).

At an operation 52, a determination is made as to whether a firstintermediate portion of the control gesture has been performed by theuser. Responsive to the first intermediate portion of the controlgesture having been performed, method 44 proceeds to an operation 54. Inone embodiment operation 52 is performed by a gesture module similar toor the same as gesture module 24 (shown in FIG. 1 and described above).

At operation 54, a second haptic stimulation is determined. The secondhaptic stimulation corresponds to the first intermediate portion of thecontrol gesture. In one embodiment, operation 54 is performed by astimulation module similar to or the same as stimulation module 26(shown in FIG. 1 and described above).

At an operation 56, the second haptic stimulation is generated for theuser. In one embodiment, operation 56 is performed by one or moreactuators similar to or the same as actuators 16 (shown in FIG. 1 anddescribed above).

Referring back to operation 52, responsive to the first intermediateportion of the control gesture not having been performed, method 44proceeds to an operation 58 at which a determination is made as towhether the control gesture has failed. This may be determined based onthe output signal of a user interface similar to or the same as userinterface 14 (shown in FIG. 1 and described above). The determination ofwhether the control gesture has failed may be made, for example, basedon an amount of time since performance of the initial portion of thecontrol gesture, if actions not included in the control gesture havebeen performed since performance of the initial portion of the controlgesture, and/or other factors. In one embodiment, operation 58 isperformed by a gesture module similar to or the same as gesture module24 (shown in FIG. 1 and described above).

Responsive to a determination at operation 58 that the control gesturehas not failed, method 44 returns to operation 52. Responsive to adetermination at operation 58 that the control gesture has failed,method 44 proceeds to an operation 60. At operation 60, hapticstimulation is determined that indicates failure of the control gesture.In one embodiment, operation 60 is performed by a stimulation modulesimilar to or the same as stimulation module 26 (shown in FIG. 1 anddescribed above).

Upon determination of the haptic stimulation at operation 60, thedetermined haptic stimulation is generated for the user at operation 62.In one embodiment, operation 62 is performed by one or more actuatorssimilar to or the same as actuators 16 (shown in FIG. 1 and describedabove).

Referring back to operation 52, further responsive to determination thatthe first intermediate portion of the control gesture has beenperformed, method 44 proceeds to an operation 64. At operation 64, adetermination is made as to whether an ending portion of the controlgesture has been performed. Responsive to a determination that thecontrol gesture has ended, method 44 ends the provision of hapticstimulation associated with the control gesture. In one embodiment, afinal haptic stimulation associated with the completion of the controlgesture is further determined and generated. In one embodiment,operation 64 is performed by a gesture module similar to or the same asgesture module 24 (shown in FIG. 1 and described above).

Responsive to a determination at operation 64 that the ending portion ofthe control gesture has not yet been performed, method 44 proceeds to anoperation 66. At operation 66, a determination is made as to whether thecontrol gesture has failed. This may be determined based on the outputsignal of a user interface similar to or the same as user interface 14(shown in FIG. 1 and described above). The determination of whether thecontrol gesture has failed may be made, for example, based on an amountof time since performance of the first intermediate portion of thecontrol gesture, if actions not included in the control gesture havebeen performed since performance of the first intermediate portion ofthe control gesture, and/or other factors. In one embodiment, operation66 is performed by a gesture module similar to or the same as gesturemodule 24 (shown in FIG. 1 and described above).

Responsive to a determination at operation 66 that the control gesturehas not failed, method 44 returns to operation 64. Responsive to adetermination at operation 66 that the control gesture has failed,method 44 proceeds to operations 60 and 62.

In one embodiment, the control gesture includes more than oneintermediate portion. In this embodiment, method 44 is expanded betweenoperations 52 and 64 to monitor the additional intermediate portion(s)and generate corresponding haptic stimulation in a manner similar tothat shown and described for the first intermediate portion.

FIG. 6 illustrates a method 68 of determining and/or providing hapticstimulation to a user. The haptic stimulation may be determined and/orprovided to the user in conjunction with a game, and/or in othercontexts. The operations of method 68 presented below are intended to beillustrative. In some embodiments, method 68 may be accomplished withone or more additional operations not described, and/or without one ormore of the operations discussed. Additionally, the order in which theoperations of method 68 are illustrated in FIG. 6 and described below isnot intended to be limiting.

In some embodiments, method 68 may be implemented in one or moreprocessing devices (e.g., a digital processor, an analog processor, adigital circuit designed to process information, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information). The one or moreprocessing devices may include one or more devices executing some or allof the operations of method 68 in response to instructions storedelectronically on an electronic storage medium. The one or moreprocessing devices may include one or more devices configured throughhardware, firmware, and/or software to be specifically designed forexecution of one or more of the operations of method 68.

At an operation 70, a view of virtual equipment is presented to the uservia a touch sensitive electronic display. The view includes selectablesections that are selectable by the user to interact with the virtualequipment shown in the view. In one embodiment, the view of the virtualequipment is determined by a content module similar to or the samecontent module 22 (shown in FIG. 1 and described above).

At an operation 72, selection of one of the selectable sections of theview of the virtual equipment is received. The selection may be receivedthrough the touch sensitive electronic display through which the view ispresented.

At an operation 74, operation of the virtual equipment is simulatedand/or one or more parameters of the virtual equipment is adjusted basedon the selection received at operation 72. In one embodiment, operation74 is performed by an equipment module similar to or the same asequipment module 28 (shown in FIG. 1 and described above).

At an operation 76, haptic stimulation associated with the operation ofthe virtual equipment and/or the adjustment of the operating parametersof the virtual equipment effected at operation 74 is determined. In oneembodiment, operation 76 is performed by a stimulation module similar toor the same as stimulation module 26 (shown in FIG. 1 and describedabove).

At an operation 78, the haptic stimulation determined at operation 76 isgenerated for the user. In one embodiment, operation 78 is performed byone or more actuators similar to or the same as actuators 16 (shown inFIG. 1 and described above).

As was mentioned above, the description of complex haptic stimulation inresponse to gesture-based control for a single user is not intended tobe limiting. Haptic stimulation generated in response to gesture-basedcontrol in a multi-user environment is also contemplated. The hapticstimulation provided to a given user may include haptic feedback that isresponsive to gesture-based controls of another user and/or otherphenomena.

FIG. 7 is a diagram illustrating an example 100 of a portable computingdevice configured in accordance with one or more aspects of the presentinvention. Particularly, a tablet computer 102 is shown. Tablet computer102 includes a screen 104 mounted to a body 106, with the top surface108 representing a surface with which a user interacts. In this example,top surface 108 is bordered by body 106, but the screen area couldextend all the way to the edges of the tablet. Tablet computer 102 canuse any number or type of touch-sensing technologies to determine when auser has touched on or near surface 106. For example, surface 108 mayinclude a resistance or capacitance-based touch sensing system and/or anoptical touch sensing system.

Although a tablet computer is shown in this example, it can beunderstood that any touch-enabled computing device can be used. Forinstance, a smaller device such as a cellular telephone or media playermay feature a touch-enabled display and can provide haptic outputs inaccordance with the present subject matter. As another example, ahaptically-enabled display surface associated with and interfaced toanother computing device (e.g., a desktop or server computer) can beused. For example, surface 108 may represent a larger touch-enabledsurface such as a table upon which a screen image is projected fromabove or below. A flat surface is depicted here, although the presentsubject matter could be applied for use in devices with curved surfaceand/or non-smooth surfaces. Gestures input to the computer (or othercomputing device) via touch may be control gestures. It will beappreciated that the description of gestures involving touch are notintended to limit the scope of control gestures discussed herein.Control gestures may include (with or without touch gestures)manipulation of a component or body (e.g., “tilt” controls),manipulation of one or more stick controls, manipulation of one or morebuttons, manipulation of one or more switches, and/or manipulation of orinteraction with other interface features.

FIG. 7 illustrates an example of the architecture of a computing device102 at 110. Computing device 102 comprises one or more processors 112configured to execute computer program modules, a memory 114 such asRAM, ROM, or other memory technology, a display interface 116, a hapticinterface 118, I/O interface 120, and network interface 122. Anysuitable display technology can be used. In some embodiments, an LCDdisplay is used.

Haptic interface 118 can comprise suitable components for driving one ormore actuators used to play back haptic effects so as to provide aphysical sensation to a user of device 102. For example, some or all ofdisplay 104 may include embedded actuators so that targeted physicaloutput can be provided to a portion of the display to provide a physicaleffect where the user touches surface 108. Additional actuators may beused to provide haptic output via other surfaces of tablet 102, such asits sides and the surface opposite surface 108 (i.e., the back of thedevice). It will be appreciated that the location of the actuatorrelative to the desired physical effect can vary. For example, in orderto produce an effect at a first part of the screen, an actuator at asecond part of the screen (or elsewhere in the device) may be driven sothat the properties of intervening components of the screen and/ordevice influence what is felt at the first part of the screen.

In one embodiment, the tablet comprises an actuator having an eccentricrotating mass motor. The actuator is coupled either directly orindirectly to a surface of the tablet housing. Powering the motor causesvibration on the surface that a user can feel. By varying the magnitudeand freq of the signal sent to the actuator, various effects arepossible. As another example, an actuator may be used to raiser or lowersections of the screen to create ridges, troughs, or other features. Asa further example, an actuator can comprise a piezoelectric. Forexample, a piezoelectric actuator can be embedded, at least partially,in an inorganic polymer matrix, such as silicone. As yet a furtherexample, an actuator may comprise a macro-fiber composite actuator orpiezocomposite actuator. Such actuators may be formed as a thin layer ofpiezoelectric fibers suspended in a matrix (e.g., epoxy). The fibers maycommunicate electrically with polyimide electrodes. Many other types ofactuators may be used, and so this exemplary description of actuators isnot meant to be limiting.

I/O interface 120 can be used by processor(s) 112 to receive input andprovide output using any suitable components. For example, I/O interface120 may link to speakers and/or a microphone for receiving voice inputand providing audio output. As another example, I/O interface 120 mayprovide a connection to peripheral devices such as a mouse or stylusused to provide input to the device, or to an imaging sensor used tocapture still images and/or video.

Network interface 122 can be used to link device 102 to a network usingone or more networking technologies. For instance interface 122 mayprovide a connection to suitable components for connecting to an IEEE802.11 (Wi-Fi) or 802.16 (Wimax) network, or a connection usingBluetooth technology. As another example, interface 122 may allowcommunication via a telephone, Ethernet, or other wired connection ormay support other wireless technology such as communication via an IRport.

Computing device 102 can comprise additional components—for example, oneor more storage components (e.g., magnetic or solid-state hard diskdrives) can be included. If computing device 102 comprises a cellulartelephone, appropriate RF components may be included as well.

Memory 114 tangibly embodies one or more program components thatconfigure computing device 102 to operate in an intended manner. Forexample, memory 114 can include one or more applications, an operatingsystem, and can also include stored data. As illustrated, memory 114also includes a program component 124 for providing an interactive gamein accordance with one or more aspects noted below.

Generally, the game can configure computing device 102 to present a playarea 126 via display 104, track the movement of a virtual object 128(e.g., a ball) in the play area, and respond to user interactions tolaunch and deflect the virtual object during play using paddle 130.Additionally, the game can be configured to play back haptic effects asthe virtual object moves through and encounters features of the playarea. The haptic effects can be selected to provide a sensation thatdiffers based on the particular features that are encountered. In thisexample, play area includes a top T, bottom B, left side L, and rightside R. The game can be configured so that the virtual object isdeflected by paddle 130 prior to reaching left side L. If multipleplayers are involved, the virtual object may exit at one or more sidesT, B, or R and pass to another user's screen as noted below.

FIG. 8A illustrates an example 200A of the use of a game to supportmulti-user play. The game program can support sending and receiving datato facilitate tracking the position of the virtual object in a play areathat comprises multiple screens. In this example a first device 202 isinterfaced to a second device 204 via a network 206. Network 206 maycomprise a local area network, a wide area network, or may represent adirect connection between devices 202 and 204. By interfacing multipledevices running an instance of the game, a user can experience hapticeffects when the virtual object encounters features in another user'sscreen. Such haptic effects may include haptic effects determined inaccordance with a control gesture input on another user's screen.

For example, if the play area 126 of FIG. 7 is used, then when virtualobject 128 exits at right side R of a first device it may enter thescreen at right side R of the second device. As another example, thedevices may have mirror-image layouts; that is, device 202 may featurepaddle 130 along left side L while device 204 includes paddle 130 alongright side R. In that case, when the virtual object reaches right side Rof the play area of the first device, it may enter the play area at leftside L of the other device, headed towards right side R and paddle 130in the play area of the other device.

Server 208 is depicted to illustrate that in some embodiments,multi-user play may be facilitated by a server. However, as noted above,in some embodiments game program is configured to directly interfacewith other instances without need of a server.

FIG. 8B illustrates another example 200B of a multi-user play. In thisexample, three devices 202, 204, and 210 are interfaced via a firstnetwork 206. A second network 214 facilitates interaction with a fourthdevice 212. For example, network 206 may comprise a local area networkconnection while network 214 may comprise a wide-area networkconnection. Compass 216 is illustrated to show that in some embodimentsthe relative position of multiple players can be considered. Forexample, as the “westmost” player, device 202 may include paddle 130 atleft side L. Device 204 may have its paddle positioned along top side T,since device 204 is “northmost.” Device 210 may have its paddlepositioned along bottom side B, while device 212 may have its paddlepositioned along right side R. Alternatively, the paddle may remain onthe same side of the screen for each user but with appropriate mappingbetween edges to maintain the relative position of the users around theshared play area.

It will be understood that in various embodiments the game can beconfigured to dynamically adjust the behavior and connections betweenthe play areas based on factors such as each device's orientation, therelative position of the players, and other considerations such as therelative size of screen areas. As will be noted below, the virtualobject can move from one screen to another in other manners in additionto or instead of encountering the boundaries of the play area.

FIG. 9 illustrates an example of play areas 302A and 302B for tworespective users A and B over a plurality of time intervals (I), (II),and (III). Each play area 302 includes a respective paddle 304. At timeinterval (I), virtual object 306 has been launched or deflected frompaddle 304B towards the boundary of play area 302B. As shown at timeinterval (II), exit point 308 from play area 302B is mapped to an entrypoint 310 in play area 302A. The mapping may allow the virtual object topass instantly between the play areas or there may be a delay based onthe distance between the players (e.g., as determined by GPS and/orother triangulation or proximity detection techniques). In any event,time interval (II) depicts virtual object 306 encountering paddle 304A.For example, user A may have moved paddle 304A by sliding his or herfingers along the display surface of his device to intercept virtualobject 306.

When virtual object 306 encounters paddle 304A, a haptic effect H1 isselected and played back. As illustrated, haptic effect H1 is localizedto the point at which user A touches paddle 304A (and/or another part ofplay area 302). As was noted above, the sensation can be generated bycommanding one or more actuators to provide motion or another effect;the actuators may be located at the point at which the effect isintended to be felt and/or elsewhere. FIG. 9 shows effect H1 as“(((H1)))” in play area 302A and as “(H1)” in play area 302B since theeffect is also played back for player B and localized to player B'stouch point. However, as indicated the intensity of effect H1 differs asbetween players A and B.

The haptic effect can be selected based on the simulated physics of thegame. For example, the paddles 304 may represent a hard surface, and soeffect H1 may comprise a strong, sharp effect. Since a deflection ismeant to represent a “hit,” effect H1 may be the strongest effect in thegame. The “hit” is played back to user B to alert user B that thevirtual object will be returned and user B can prepare to deflect theincoming virtual object. The hit may be played back to player B with asuitable indicator of direction—for instance, the effect may be designedto feel like it originated from the left, rather than the top or bottom;this may be helpful when three of more users are playing together.

By playing back the haptic effect even for a collision (or other event)occurring in a different player's screen, the game can enhance theperception that the players are sharing a space even though the playerscannot view one another's play areas. Thus, the players may become moreimmersed in the game and may have a more compelling game experience.

FIG. 10 illustrates another example of play and depicts the virtualobject ricocheting off the border of a play area. Particularly, threetime intervals (I), (II), and (III) are again shown. Play areas 402A and402B correspond to players A and B, while paddles 404 and virtual object406 are also illustrated. As shown at time interval (I), virtual object406 is launched with a trajectory towards a point 408 at the topboundary of play area 402. Interval (II) illustrates when virtual object406 encounters point 408. A “bounce” haptic effect H2 is played back toplayers A and B, localized to their touch points at respective paddles404A and 404B.

Since the “bounce” occurs in play area 402B and is at a closer distanceto paddle 404B than paddle 404A, it is depicted as “((H2))” in play area402B and “(H2)” in play area 402A since the bounce is “louder” forplayer B. As shown at time interval (III), after the bounce the virtualobject passes to play area 402A. Alerted to the bounce, player A mayattempt to intercept the virtual object and prevent it from reaching thegoal area behind paddle 404A.

FIG. 11 is a diagram illustrating another aspect of the present subjectmatter. In addition to or instead of haptic effects played in responseto events changing the virtual object trajectory or other “discrete”events, haptic effects can be played back to simulate a continuouseffect. In this example, play areas 502 for two players A and B areshown in a time interval (I) and a time interval (II). For purposes ofillustration, play areas 502 are shown in a “landscape,” rather than“portrait” orientation. Each play area features a respective paddle 504as well, and virtual object 506 is depicted.

Each play area of this example also includes seams 508 represented bydotted lines. For example, seams 508 may represent boundaries betweenplanks in a wooden surface depicted in the play area. To simulate a woodpanel back ground, a continuous low rumble effect to correlate with thevirtual object rolling across the surface can be combined with clickeffects to correlate with the virtual object encountering seams 508.This effect is shown as “H3” in FIG. 11. At time interval (I), theeffect is shown as “((H3))” for players B and “(H3)” for player A sincethe virtual object is closer to paddle 504B than paddle 504A. At timeinterval (II), effect H3 is louder for paddle 504A since virtual object506 is moving towards player A. Although the background effect is shownin conjunction with seams 508, a background effect could be included tosimulate a surface alone (i.e. a continuous surface) or could vary asthe simulated background surface changes (e.g., from a wood area to ametal area to a concrete area, etc.).

FIG. 12 is a flowchart illustrating illustrative steps in a method 600for providing a game in accordance with the present subject matter.Block 602 represents setting up one or more play areas. For example, iftwo users desire to play, respective play areas can be initialized andmappings between the shared boundaries (and/or other entry-exit points)can be determined.

Block 604 occurs while play continues. At least one instance of the gamecan track the position and motion of the virtual object based oninteraction with paddles, obstacles, and characteristics of the playarea based on a model simulating physics of the game. For example, themodel can provide for changes in the virtual object's speed anddirection based on simulating momentum, mass, and materialcharacteristics of the virtual object and the other items in the playarea.

At block 606, based on the position and motion of the virtual object at,before, and/or after an event, one or more haptic effects to play backcan be determined. For example, if the virtual object encounters aboundary or other object in the play area, a haptic effect associatedwith the physical interaction between the virtual object andboundary/object can be selected for playback. Differentboundaries/objects may result in different effects. For example, aborder or paddle may result in a “hard” effect, while obstacles includedin the play area may have “soft” effects. Simulated properties of thevirtual object can be taken into account as well—the game may support amode with a hard (e.g., steel) virtual object or a soft (e.g., rubber)virtual object with appropriate changes in the haptic output scheme.

Additionally or alternatively the haptic effect can relate to abackground effect. For instance, as was noted above, a continuous hapticeffect simulating passage of the virtual object over a simulated surfacecan be provided based on characteristics of that surface. As anotherexample, the surface may include material or an obstacle for the virtualobject to pass through and a suitable haptic effect can be provided tosimulate passage through the material/obstacle.

At block 608, the game determines the position of the virtual objectrelative to the haptic delivery point(s) to adjust how the haptic effectis to be output. For instance, a haptic delivery point can include thepoint at which a user touches the screen of a device. The “loudness”(i.e., intensity) of the haptic effect can be inversely proportional tothe distance between the delivery point and the virtual object.Directionality may also be applied. For example, if a ricochet occurs onanother screen, the haptic effect that is presented may include adirectional component or may be otherwise be presented to give anindication of where the ricochet occurred.

At block 610, suitable signals are sent to the actuators to generate thehaptic effect(s) having the desired volume. For example, the game canconsult a library of signal patterns for use in generating differenthaptic effects and use the signal patterns to command one or moreactuators embedded in the screen and/or other portions of the device.The haptic effects may include sounds and/or visual elements as well.

For multi-user play, each respective instance of the game can determinethe virtual object's position and motion while in that instance's playarea and pass that information to the other instances. When the virtualobject exits the play area, information regarding the virtual object'smotion (e.g., a vector with direction and speed) can be used to continuetracking by the instance of the game whose play area is to receive thevirtual object.

In some embodiments, when an event occurs and/or when background effectsare provided, the haptic effect is selected by the instance of the gamewhose play area contains the virtual object when the haptic effect is tobe triggered and that information is provided to the other instances ofthe game. For example, in a game involving player A and player B, if thevirtual object collides with an obstacle, border, or paddle in playerA's play area, the instance of the game on player A's device can provideplayer B's device the desired haptic effect along with information aboutthe collision and position thereof for use by the instance of the gameon player B's device in determining a volume or directionality for theeffect.

FIG. 13 is a diagram illustrating an example of an interface 700 for aninstance of a game configured in accordance with aspects of the presentsubject matter. In this example, a pinball-like game is presented inwhich the play area 702 includes a border 704, 706 that extends inwardfrom the screen boundaries. The objective of the game is to prevent avirtual object (not shown), such as a ball or other object, fromreaching 708 by deflecting the virtual object using paddle 710. To beginplay, the virtual object may be launched from paddle 710 or may appearelsewhere in play area 702.

Interface 700 includes a plurality of control buttons 712, 714, 716, and718 which may be used to provide inputs and access menus. For example,buttons 712 and 714 may comprise play and pause buttons, while button716 provides a “launch” command and button 718 exits the game orlaunches a menu for configuring, saving, or exiting the game. In someembodiments, control buttons 712-718 may be provided with suitablehaptic effects to simulate pushing a physical button.

Turning to play area 702, a plurality of simulated lights 720 can beprovided to enhance the visual experience; the lights may or may not actas obstacles. A bumper 722 may cause the virtual object to “bounce” inresponse to a collision in a manner different from a ricochet fromborder 704, 706. For example, border 704, 706 may be presented as asimulated metal border which causes a sharp ricochet effect. Bumper 722may feature an initial amount of “give” before imparting force on thevirtual object in a manner similar to that of a pinball machine bumper.Accordingly, when the virtual object encounters border 704, 706 orbumper 722, different respective haptic effects may be played back atthe point(s) at which the user contacts the screen to provide differentsensations in accordance with the simulated physical response of theborders/bumper. Additionally, as was noted above, the intensity of theeffect can depend on distance from the point(s) at which the usercontacts the screen.

This example features metal bars 724, 726, and 728, which may provide astill further response to collisions with a virtual object and may beassigned their own respective haptic effect. Arrows 730 may comprise avisual effect and/or may result in an acceleration of the virtual objectinto xylophone structure 732. In this example, xylophone structure 732comprises a plurality of keys (732A, 732B, 732C identified), with eachascending key having its own associated haptic effect. For example, asthe virtual object moves from key 732A to 732B to 732C, the hapticeffect may increase in pitch along with corresponding xylophone soundeffects. At the same time, the haptic effect may decrease in intensityas the virtual object moves away.

In some embodiments, point 734 represents an exit from the play area andinto a second user's play area, which is identical to play area 702.When the virtual object enters play area 702, it may be returned viachute 736 along with an accompanying “rattle” representing passagethrough the chute. As was noted above, each haptic effect in one user'splay area can also be played back in the other user's (or users') playarea(s) but with a corresponding decrease in intensity based on theseparation from the site of the event causing the haptic effect and theuser's point of contact with the screen.

Some embodiments feature one or more instances of a vortex as shown at738. Vortex 738 can comprise a portion of play area 702 that attractsthe virtual object toward opening 740. If the virtual object reachesopening 740, the virtual object may pass into another play area. Whenthe virtual object initially contacts vortex 738, a first haptic effectrepresenting the “pull” of the vortex may be played back, with theeffect becoming stronger until (and if) the virtual object reachesopening 740. At that point, an “exit” effect may be played back torepresent the virtual object's exit from the vortex in another playarea. This can, for instance, alert the user of the play area receivingthe virtual object to move his or her paddle 710 into position.

In some embodiments, if the virtual object is not deflected from goalarea 708, a haptic effect representing entry of area 708 is presented,such as absorption of the virtual object or an explosion. At that point,depending upon the rules of the game, the virtual object may be providedfor launch again via paddle 710 for another round. In some embodiments,the game proceeds until one player reaches a predetermined score level(e.g., 7 goals) and/or score differential (ahead by 3 goals). As anotherexample, point values may be associated with hitting certain obstacles(e.g., bumper 722, bars 724, 726, 728, lights 720) or passing throughall keys of xylophone structure 732. Embodiments may support moving ordestroying obstacles (e.g., breaking through bricks) during the courseof play, with suitable haptic effects provided based on motion ordestruction of the obstacles.

In some embodiments, a computing device may be sensitive to distance ofa user's fingers or stylus from the touch screen and/or touch pressure.These features may be used during play of the game and/or in configuringthe game application. For instance, if the device indicates that theuser is not touching the screen or another haptically-enabled area,haptic effects can be turned off to reduce power consumption. As anotherexample, the user may be able to hover to provide input. For instance,the area at which buttons 712-716 are presented may ordinarily appear asbrushed metal, but the buttons may appear in response to a user hoveringor touching the area.

In several examples above, game play proceeded based on movement of apaddle using touch. Additionally or alternatively, gameplay may dependon tilt sensors and/or accelerometers. For example, users may be able totilt or swing their devices to affect the movement of the virtual objectand/or paddle position. Haptic effects can be delivered at the point(s)at which the users grip their respective devices. In some embodiments,the game may use multiple paddles per user or may use no paddles at all,with all input based on tilt/acceleration.

Several examples of multi-user play were provided. In some embodiments,single-user play is also supported. For example, a play area may becompletely closed, with the virtual object returning towards the paddleand goal area. As another example, single user play may proceed with oneor more other players simulated, with the other players having arespective simulated play area and with corresponding haptic feedbackwhen the virtual object enters the simulated play area.

Although the invention has been described in detail for the purpose ofillustration based on what is currently considered to be the mostpractical and preferred embodiments, it is to be understood that suchdetail is solely for that purpose and that the invention is not limitedto the disclosed embodiments, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present invention contemplates that, to the extent possible, one ormore features of any embodiment can be combined with one or morefeatures of any other embodiment.

What is claimed is:
 1. A method of producing a haptic effect,comprising: monitoring a motion of a body part of a user received from auser interface device with a sensor; comparing the motion of the bodypart to a control gesture with a gesture module executable by aprocessor; generating a first haptic effect with a haptic output deviceif the motion of the body part corresponds to the control gesture; andgenerating a second haptic effect with the haptic output device if themotion of the body part does not correspond to the control gesture, thesecond haptic effect being different from the first haptic effect. 2.The method of claim 1, wherein the body part is an appendage of theuser.
 3. The method of claim 1, wherein the body part is a head of theuser.
 4. The method of claim 1, wherein the motion of the body partcomprises changing a direction of motion of the body part.
 5. The methodof claim 1, wherein the motion of the body part comprises changing anorientation of the body part.
 6. The method of claim 1, whereinmonitoring the motion of the body part comprises monitoring anacceleration of the body part.
 7. The method of claim 1, furthercomprising determining the first haptic effect to be generated based oncontent being provided to the user via the user interface device with acontent module executable by the processor.
 8. A haptic effect enabledsystem, comprising: a user interface device; a sensor configured tomonitor a motion of a body part of a user received from the userinterface device; a haptic actuator configured to generate a hapticeffect; and a processor in operative communication with the userinterface, the sensor and the haptic actuator, the processor comprisinga gesture module electronically coupled to the sensor, the gesturemodule configured to compare the motion of the body part with a controlgesture, and a stimulation module electronically coupled to the gesturemodule and the haptic device, the stimulation module configured todetermine the haptic effect to be generated by the haptic actuator ifthe motion of the body part corresponds to the control gesture.
 9. Thesystem of claim 8, wherein the stimulation module is further configuredto determine a second haptic effect to be generated by the hapticactuator that is different from the haptic effect if the motion of thebody part does not correspond to the control gesture.
 10. The system ofclaim 8, wherein the body part is an appendage of the user and the userinterface device is configured to be carried by the appendage of theuser.
 11. The system of claim 8, wherein the body part is a head of theuser and the user interface device is configured to be carried by thehead of the user.
 12. The system of claim 8, wherein the motion of thebody part comprises changing a direction of motion of the body part. 13.The system of claim 8, wherein the motion of the body part compriseschanging an orientation of the body part.
 14. The system of claim 8,wherein the sensor is configured to monitor an acceleration of the bodypart.
 15. The system of claim 8, wherein the motion of the body partcomprises an initial portion, an intermediate portion and an endingportion, wherein the haptic effect is generated if the initial portionof the motion corresponds to an initial portion of the control gesture,and wherein a second haptic effect that is different from the hapticeffect is generated if the intermediate portion of the motioncorresponds to an intermediate portion of the control gesture.
 16. Thesystem of claim 15, wherein a third haptic effect that is different fromthe haptic effect and the second haptic effect is generated if theending portion of the motion corresponds to an ending portion of thecontrol gesture.
 17. The system of claim 8, wherein the user interfacedevice is configured to provide content to the user, and wherein thestimulation module is further configured to determine the haptic effectto be generated based on the content provided to the user.
 18. A methodof producing a haptic effect, comprising: monitoring a motion of a bodypart of a user received from a user interface device with a sensor,wherein the motion of the body part comprises an initial portion, anintermediate portion and an ending portion; comparing the motion of thebody part to a control gesture with a gesture module executable by aprocessor; generating a first haptic effect with a haptic output deviceif the initial portion of the motion corresponds to an initial portionof the control gesture; and generating a second haptic effect with thehaptic output device if the intermediate portion of the motioncorresponds to an intermediate portion of the control gesture, whereinthe second haptic effect is different from the first haptic effect. 19.The method of claim 18, further comprising: generating a third hapticeffect with the haptic output device if the ending portion of the motioncorresponds to an ending portion of the control gesture, wherein thethird haptic effect is different from the first haptic effect and thesecond haptic effect.